Thank you! I'll take a look asap.

thanks for this.
I think we should have more discussion about the API.
If the additional data is at unit level we just could add some properties in the sorting object this would make more sens.
If the additional data is at spike level we would need an additional function at API level not do this extractor per extractor.

If the additional data is at spike level we would need an additional function at API level not do this extractor per extractor.

This is what I was wondering about. The easiest solutions for now would be to return matching arrays per unit, but for the sake of reusability I guess the BaseSorting class would need to be able to handle arbitrary per-spike properties.

In the case of HerdingSpikes, a lot of useful data is already computed or extracted : waveforms, locations and amplitudes are already included, PCA is also computed but not included in the output. All of this is then recomputed by the sorting analyzer. Then it would be up to individual sorting extractors to match the expected data structure.

On the data HS extracts:

• Amplitude: This is not the real amplitude, but a re-scaled version. I don't think these are useful.
• Waveforms: These are peak channel waveforms, internally cached memmapped for quick PCA computation and can be written out into the hdf5 file the extractor reads.
• Channel index: Useful as these are peak channels.
• Unit index: Internal use.
• x and y coordinates: COM based estimates, very useful as they are very quick to estimate.

I feel peak channel and x/y locations could be put into a SortingAnayzer object, that's where they would normally be found. Would this be possible?

I agree with Sam that the unit properties should be written as such. I would be curios about how to handle spike properties if you have them so tagging along here.

If this is too much for now, I think the PR can stay as is and just expose unit locations as a property.

Regarding individual spikes, being able to load arbitrary properties on top of spike times might require changes deep in the structure. A simpler solution might be for individual extractors to specify "pre-computed" extensions that are then loaded when the sorting analyzer is created.

If the changes can be mostly contained at the extractor level, I think it could avoid some extra abstractions at the sorter level ?

This might not be relevant for most use-cases but for instance, computing waveforms on BioCAM data takes about 30 minutes with 100 spikes per unit, and the binary weighs about 180 GB. Meanwhile, the data is already on the HDF5 file read by the extractor, with a total size of less than 200 MB (same dtype, roughly the same cutout times).